Adding images to a text based electronic message

ABSTRACT

A plurality of word groups that satisfy at least one criterion, each word group comprising at least one word, can be identified in an electronic message. For each word group that satisfies the at least one criterion, at least a first image corresponding to a meaning of the word group can be automatically selected from a plurality of images. Each image selected for each respective word group that satisfies the at least one criterion can be presented with the electronic message.

BACKGROUND

The present invention relates to electronic communication and, morespecifically, to electronic messages.

Various types of electronic communication used to communicate electronicmessages include electronic mail (e-mail), text messaging, instantmessaging and posting on websites. The use of these types of electronicmessages has grown significantly in recent decades as use of theInternet and mobile devices has grown. Indeed, in many countriescommunication electronic messages now takes place more frequently thanvoice communication.

Various communication protocols have been developed to facilitate thecommunication of electronic messages. Such protocols include the PostOffice Protocol (POP), Internet Message Access Protocol (IMAP) andMessaging Application Programming Interface (MAPI), which support e-mailcommunication. Other protocols commonly used for communication ofelectronic messages include the Simple Mail Transfer Protocol (SMTP),the Simple Message Protocol (SMP), and the Short Message Service (SMS).Also, web sites (e.g., social networking sites, collaboration sites, webforums and blogs) accept electronic messages as posts, and activitystreams are used to present multiple posts relevant to a person, groupor topic.

SUMMARY

A method includes identifying in a first electronic message a pluralityof word groups that satisfy at least one criterion, each word groupcomprising at least one word. The method also includes, for each wordgroup that satisfies the at least one criterion, automaticallyselecting, using a processor, from a plurality of images at least afirst image corresponding to a meaning of the word group. The methodalso includes presenting with the first electronic message each imageselected for each respective word group that satisfies the at least onecriterion.

A system includes a processor programmed to initiate executableoperations. The executable operations include identifying in a firstelectronic message a plurality of word groups that satisfy at least onecriterion, each word group comprising at least one word. The executableoperations also include, for each word group that satisfies the at leastone criterion, automatically selecting from a plurality of images atleast a first image corresponding to a meaning of the word group. Theexecutable operations also include presenting with the first electronicmessage each image selected for each respective word group thatsatisfies the at least one criterion.

A computer program includes a computer readable storage medium havingprogram code stored thereon. The program code is executable by aprocessor to perform a method. The method includes identifying, by theprocessor, in a first electronic message a plurality of word groups thatsatisfy at least one criterion, each word group comprising at least oneword. The method also includes, for each word group that satisfies theat least one criterion, automatically selecting, by the processor, froma plurality of images at least a first image corresponding to a meaningof the word group. The method also includes presenting, by theprocessor, with the first electronic message each image selected foreach respective word group that satisfies the at least one criterion.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts an example of an electronic message and a plurality ofimages associated with the electronic message.

FIG. 2 depicts another example of an electronic message and a pluralityof images associated with the electronic message.

FIG. 3 depicts another example of an electronic message in which wordgroups are replaced with corresponding images.

FIG. 4 depicts example of an activity stream in which a plurality ofelectronic messages and images associated with the electronic messagesare presented.

FIG. 5 is a block diagram illustrating an example of a computingenvironment.

FIG. 6 is a flow chart illustrating an example of a method of presentingwith electronic messages images selected for respective word groupscontained in the electronic messages.

FIG. 7 is a block diagram illustrating example architecture for aprocessing system.

DETAILED DESCRIPTION

The present invention relates to electronic communication and, morespecifically, to electronic messages. In accordance with the inventivearrangements disclosed herein, one or more word groups that satisfy atleast one criterion can be identified in an electronic message (e.g., ane-mail, a text message, an instant message or a post to a blog, forum,social networking system, collaboration system or the like). Each wordgroup can include one or more words. For each word group that satisfiesthe at least one criterion, at least one image that corresponds to ameaning of the word group can be selected. The image(s) selected can bepresented with the electronic message, for example responsive to a useraccessing and viewing the electronic message. Accordingly, rather thanreading the text of the electronic message, a user can view the image(s)to gain a quick understanding of the information contained in theelectronic message.

Moreover, the processes described can be applied to a plurality ofelectronic messages, for example electronic messages presented in anactivity stream. Rather than reading the text of each of the electronicmessages posted in the activity stream, the user can quickly view one ormore images associated with each electronic message to quickly gain anunderstanding of the various electronic messages. This can save the usera significant amount of time when reviewing multiple electronicmessages.

Several definitions that apply throughout this document now will bepresented.

As defined herein, the term “image” means a reproduction or imitation ofthe form of one or more persons or things. Examples of an image includea picture and a drawing. A button, icon or menu item that is userselectable to initiate a method, process or function to be performed bya data processing system is not an “image” as the term “image” isdefined herein.

As defined herein, the term “word group” means a human language word,phrase, sentence or paragraph. As defined herein, punctuation (e.g., agroup of one or more periods, commas, colons, semicolons, dashes, etc.)and non-letter characters are not “word groups.” Nonetheless, a word,phrase, sentence or paragraph may include punctuation and/or non-lettercharacters.

As defined herein, the term “human language” is a language spoken orwritten by human beings that is not a computer programming language. A“human language” may be referred to as a “natural language.”

As defined herein, the term “electronic message” means a message thatincludes text and which is communicated electronically for presentationto at least one user. Examples of an “electronic message” include ane-mail, a text message, and an instant message. A further example of an“electronic message” includes a post to a blog, forum, social networkingsystem, collaboration system or the like. An “electronic message” may,but need not, include pictures, images, multimedia, hyperlinks, and thelike.

As defined herein, the term “activity stream” means a list of electronicmessages, each of which are relevant to a person, group and/or topic.

As defined herein, the term “social networking system” means a systemthat supports communication between members of a social network.Examples of a social networking system include an Internet forum, ablog, and a social networking site. As defined herein, an “e-mailsystem,” per se, is not a social networking system. As defined herein,an “e-mail server,” per se, is not a social networking system. Asdefined herein, an “e-mail client,” per se, is not a social networkingsystem.

As defined herein, the term “responsive to” means responding or reactingreadily to an action or event. Thus, if a second action is performed“responsive to” a first action, there is a causal relationship betweenan occurrence of the first action and an occurrence of the secondaction, and the term “responsive to” indicates such causal relationship.

As defined herein, the term “computer readable storage medium” means astorage medium that contains or stores program code for use by or inconnection with an instruction execution system, apparatus, or device.As defined herein, a “computer readable storage medium” is not atransitory, propagating signal per se.

As defined herein, the term “processor” means at least one hardwarecircuit (e.g., an integrated circuit) configured to carry outinstructions contained in program code. Examples of a processor include,but are not limited to, a central processing unit (CPU), an arrayprocessor, a vector processor, a digital signal processor (DSP), afield-programmable gate array (FPGA), a programmable logic array (PLA),an application specific integrated circuit (ASIC), programmable logiccircuitry, and a controller.

As defined herein, the term “automatically” means without userintervention.

As defined herein, the term “user” means a person (i.e., a human being).

FIG. 1 depicts an example of an electronic message 100 and a pluralityof images 110, 112, 114 associated with the electronic message 100 inaccordance with the present arrangements. The electronic message 100 canbe presented on a client device to a user. In this example, theelectronic message 100 includes a plurality of word groups 102, 104. Theword group 102 can be a main topic (e.g., “Server Error”) and the wordgroup 104 can be a comment (e.g., “You are not authorized to use theserver”). The message can be processed by a data processing system(hereinafter “processing system”) executing an image selectionapplication, as will be described herein, to automatically identify theword groups 102, 104, each including at least one word, in theelectronic message 100 which satisfy at least one criterion. Examples ofa criterion include, but are not limited to, a topic, a tag, a wordtype, a relevancy, a user interest, an alert, a condition, aninstruction to a user, an action to be taken, an action that has beentaken, and the like.

The processing system also can automatically determine the meaning ofeach word group 102, 104. To determine the meaning of each word group102, 104, the processing system can implement natural languageprocessing (NLP) and semantic analysis on each word group. NLP is afield of computer science, artificial intelligence and linguistics whichimplements computer processes to facilitate interactions betweencomputer systems and human (natural) languages. NLP enables computers toderive computer-understandable meaning from natural language input. TheInternational Organization for Standardization (ISO) publishes standardsfor NLP, one such standard being ISO/TC37/SC4. Semantic analysis is theimplementation of computer processes to generate computer-understandablerepresentations of natural language expressions. Semantic analysis canbe used to construct meaning representations, semanticunderspecification, anaphora resolution, presupposition projection andquantifier scope resolution, which are known in the art. Semanticanalysis is frequently used with NLP to derive computer-understandablemeaning from natural language input.

Having determined the meaning of each word group 102, 104 the processingsystem can automatically select one or more images 110-114 correspondingto such meanings and present the selected images 110-114 with theelectronic message 100. For instance, the processing system can selectthe images 110-114 from an image database. The image database caninclude a plurality of images and, for each image, metadata. Theprocessing system can scan the metadata to identify metadata thatcorrelates to the determined meanings of the word groups 102, 104, andselect the images to which such metadata is assigned or otherwiseassociated.

In illustration, the processing system can select the image 110, whichcorresponds to the meaning of the word group 102 (e.g., the main topic).In this example, since there is a server error, the user should contactsupport. Thus, the image 110 that is selected can be an image thatindicates such. The processing system also can select the images 112,114 to correspond to the word group 104 (e.g., the comment). Forexample, since the user is not authorized to use the server, the servermay be locked for that user. Accordingly, the image 112 that is selectedcan indicate this condition. Further, in order for the user to use theserver, the server may need to be unlocked for the user, as indicated bythe selected image 114. By viewing the images 110-114, the user caneasily glean a meaning of the electronic message 100.

The images 110-114 can be presented to the user with the electronicmessage 100 in a suitable manner. For instance, in one arrangement, theimages can be presented in a tree structure 120 that presents theselected images 110-114 below the electronic message 100, therebyindicating that the images 110-114 correspond to the electronic message100. In one arrangement, the tree structure 120 can include lines 122pointing from the electronic message 100 to the images 110-114, forexample as shown. In this regard, the tree structure 120 can representand/or indicate a hierarchical relationship between the electronicmessage 100 and the images 110-114 (e.g., indicate that the images110-114 have been selected for the electronic message 100, but were notoriginally included in the electronic message 100 when the electronicmessage 100 was created). Thus, the user will be aware of suchcircumstance.

FIG. 2 depicts another example of an electronic message 200 and aplurality of images 210, 212, 214 associated with the electronic message200 in accordance with the present arrangements. The electronic message200 can be presented on a client device to a user. The images 210, 212can be selected to correspond to the meanings of the respective wordgroups 202, 204, as previously described for FIG. 1. In this example,however, a consolidated image 230 also can be presented to the user withthe electronic message 200. The consolidated image 214 can be created bythe processing system by selecting a plurality (i.e., at least two)images 240, 242, 244, 246, 248 and combining the images 240-248 into theconsolidated image 230. In this example, the images 240-248 canpictorially indicate actions that need to be taken to result in theserver being unlocked for the user.

To choose the images 240-248 to be used as components of theconsolidated image 230, the processing system can, based on the meaningsdetermined for the word groups 202, 204, access a database thatcorrelates actions, results and/or relationships with such meanings Inthis example, the meaning of the word group 204 can be processed todetermine which actions are required to unlock the server for the user,and the processing system can select images 212-244 corresponding tosuch actions. The processing system also can select an image 246representing the result of the server being unlocked, and select animage 248 to show the relationship of the result to the actions. Theprocessing system can piece together the selected images 240-248, forexample by placing in the consolidated image 230, in order, each of theimages 240-244 representing actions to be taken, placing near the images240-244 (e.g., below) the image 246 representing the desired result, andplacing the image 248 to show the relationship of the result to theactions. Thus, in this example, a user viewing the images 210, 214, 230will easily comprehend, merely by viewing the images 210, 214, 230, thatthe user should contact support because the server is locked and actionsneed to be taken by support to unlock the server.

FIG. 3 depicts another example of an electronic message 300 in whichword groups are replaced with corresponding images in accordance withthe present arrangements. The electronic message 300 can be presented ona client device to a user. In this example, a user can create theelectronic message 300, and the processing system can replace one ormore words of word groups in the electronic message 300 with images. Inillustration, the user can create an electronic message that includesthe text “I got this message indicating ‘Server Error: You are notauthorized to use the server’. I contacted support. Database was locked.Database is now unlocked.”

The processing system can identify the meaning of the word groups “I gotthis message indicating ‘Server Error: You are not authorized to use theserver’” and, based on the meaning, select an image 310 representing themeaning of the words “message indicating ‘Server Error: You are notauthorized to use the server’.” The processing system can replace thewords “message indicating ‘Server Error: You are not authorized to usethe server’” with the image 310 and replace those words with the image310. Further, the processing system can identify the meaning of the wordgroup “I contacted support” and, based on the meaning, select an image312 representing the meaning of the words “support.” The processingsystem can replace the word “support” with the image 312. The processcan continue for the word groups “Database was locked” and “Database isnow unlocked” to replace the word “locked” with an image 314 and replacethe word “unlocked” with an image 316.

In one arrangement, the processing system can position each word groupin a separate line, and insert the images 310-316 in the respectivelines. Further, the processing system can present the remaining text andthe images 310-316 in a tree structure 320. For example, the processingsystem can identify one of the word groups, for example “I got thismessage indicating ‘Server Error: You are not authorized to use theserver’,” as a main topic. The processing system can present the text “Igot this” and the image 310 at the top of the electronic message 300,and present the remaining text and images 312-316, respectively, belowthe text “I got this” and image 310, for example as shown in FIG. 3, toindicate a hierarchical relationship between the main topic and theother comments. In one arrangement, the tree structure 320 can includelines 322 pointing from the main topic text “I got this” to the text “Icontacted,” “Database was” and “Database is now,” for example as shown.This arrangement can help a user viewing the electronic message 300comprehend the electronic message 300 quickly upon viewing theelectronic message 300.

FIG. 4 depicts example of an activity stream 400 in which a plurality ofelectronic messages and images associated with the electronic messagesare presented in accordance with the present arrangements. The activitystream 400 can be presented on a client device to a user. The activitystream 400 can include a plurality of electronic messages 410, 412, 414,416, each of which includes one or more word groups 420, 422, 424, 426.The processing system can determine the meaning of each of the wordgroup 420-426 and, based on their meanings, select at least one imagecorresponding to at least one word group 420-426 in each electronicmessage 410-416. In illustration, the processing system can select animage 430 corresponding to the meaning of the word group 420, an image432 corresponding to the meaning of the word group(s) 422, an image 434corresponding to the meaning of the word group(s) 424, and an image 436corresponding to the meaning of the word group(s) 426. The processingsystem can present the images 430-436 in the activity stream 400 in amanner that associates each of the images 430-436 with the respectiveelectronic messages 410-416 containing the word groups 420-426 to whichthe images 430-436 correspond. For example, the image 430 can bepresented next to, or below, the electronic message 410, the image 432can be presented next to, or below, the electronic message 412, theimage 434 can be presented next to, or below, the electronic message414, and the image 436 can be presented next to, or below, theelectronic message 416. In another arrangement, the images 430-436 canbe presented in the respective electronic messages 410-416. Regardlessof where the images 430-436 are presented, a user can view the images toquickly glean a meaning of the electronic messages 410-416 contained inthe activity stream 400.

FIG. 5 is a block diagram illustrating an example of a computingenvironment 500. The computing environment 500 can include theprocessing previously described. In this example, the processing systemcan be implemented as an electronic message system 510, which canperform the operations described with respect to FIGS. 1-4 to associateone or more images with electronic messages 550. The computingenvironment also can include an image database 520 and one or moreclient devices 530, 532, 534.

The electronic message system 510 can be an e-mail system, an instantmessaging system, a text messaging system, a social networking system, acollaboration system, a system hosting a web forum, a system hosting ablog, or the like. In this regard, the electronic message system 510 caninclude at least one processor and memory configured to receiveelectronic messages from the client devices 530-534. The electronicmessage system 510 also can include an electronic message application(or service) 512 executed by the processor(s) and memory to performelectronic messaging processes. Examples of the electronic messageapplication 512 include, but are not limited to, an e-mail serverapplication, a text message server application, an instant messageserver application, a social networking application, a collaborationapplication, a web forum hosting application, a blog hostingapplication, and the like. The electronic message system 510 further caninclude an image selection application (or module) 514 which, whenexecuted by the electronic message system 510, determines the meaning ofword groups contained in electronic messages (e.g., using NLP andsemantic analysis), selects images based on those meanings, and presentsthe images with electronic messages, for example as previouslydescribed.

The image database 520 can include a plurality of images. As noted,within the image database 520, metadata can be assigned to, or otherwiseassociated with, each image. The metadata for each image can indicate acontext of the image, image attributes, and any other data pertinent tothe image.

Each client device 530-534 can be implemented as a processing systemincluding at least one processor and memory configured to generateelectronic messages. Examples of the client devices 530-534 include, butare not limited to, workstations, desktop computers, mobile computers,laptop computers, tablet computers, netbook computers, smart phones,personal digital assistants, set-top boxes, gaming devices, networkappliances, and the like. The electronic messages can be generated andcommunicated to the electronic message system 510 automatically or atthe behest of a user. For example, a user can interact with one or moreapplications executing on a client device 530-534 to generate and sendelectronic messages. Examples of such applications include, but are notlimited to, an e-mail client, a text messing client, an instantmessaging client, a mobile application, a web browser, and the like.

Optionally, a server can function as a client device with regard tosending and/or receiving electronic messages that are routed through theelectronic message system 510. For instance, a user of a first clientdevice 530 can interact with a server, which in this example can beidentified as the client device 534, requesting information from theserver. The server can send to the client device 530 an electronicmessage that is routed through the electronic message system 510.

The client devices 530-534 can be communicatively linked to theelectronic message system 510 via one or more networks 540. Via thenetwork(s) 540, the client devices 530-534 can send and receiveelectronic messages, which may be routed through the electronic messagesystem 510. Similarly, the electronic message system 510 can becommunicatively linked to the image database 520 via the network(s) 540.The image database 520, for example, can be stored to acomputer-readable storage medium on a server hosting the image database520. In another arrangement, the image database 520 can be stored to acomputer-readable storage medium within or communicatively linked to theelectronic message system 510 and hosted by the electronic messagesystem 510.

The network 540 is the medium used to provide communications linksbetween the various client devices 530 and the electronic message system510 and, optionally, the electronic message system 510 and the imagedatabase 520, connected together within the computing environment 500.The network 540 may include connections, such as wire, wirelesscommunication links, or fiber optic cables. The network 540 can beimplemented as, or include, any of a variety of different communicationtechnologies such as a wide area network (WAN), a local area network(LAN), a wireless network, a mobile network, a Virtual Private Network(VPN), the Internet, the Public Switched Telephone Network (PSTN), orthe like.

In operation, an electronic message 550 can be generated by a clientdevice, for example the client device 530, and communicated to theserver. In an arrangement in which the electronic message 550 is ane-mail, text message or instant message, the electronic message can beaddressed to a recipient, for example a user of the client device 532.The electronic message system 510 can process the electronic message 550by associating one or more images with the electronic message 550, forexample as previously described, and present the electronic message withthe images 552 to the user via the client device 532 (e.g., theelectronic message system 510 can communicate the electronic messagewith the images 552 to the client device, which may display theelectronic message with the images 552 to the client device on adisplay). In an arrangement in which the electronic message 550 is apost to a social network, a collaboration space, a web forum, a blog, orthe like, the electronic message system 510 can associate one or moreimages with the electronic message 550 prior to the electronic message550 being posted or after the electronic message 550 is posted. Inillustration, the electronic message system 510 can associate one ormore images with the electronic message 550 in response to presentingthe electronic message 550 in an activity stream. In this regard, theelectronic message system 510 can associate one or more images with aplurality of electronic messages presented in the activity stream, andpresent the activity stream to the user via the client device 532.

When selecting images to present with an electronic message 550, theimage selection application 514 can identify in the electronic message550 one or more word groups that satisfy at least one criterion. In onearrangement, the at least one criterion can be user defined or userselectable from a list of criteria. For example, the image selectionapplication 514 can, via a user interface of an application or webbrowser executed on a client device 530-534, present an electronicmessage configuration menu. From that menu, the user can define orselect at least one criterion for use when processing electronicmessages to determine which word groups satisfy the at least onecriterion. Via the menu, the user also can specify criteria for thetypes of images to be presented, desired image attributes for images tobe presented (e.g., minimum and/or maximum image size, maximum and/orminimum image file size, etc.). The images selected by the imageselection application 514 can be selected based on the user specifiedcriteria. Via the menu, the user also can specify a minimum and/ormaximum number of images to present per electronic message, textabbreviation rules, whether to replace words of electronic messages withimages, where the images are to be presented relative to the electronicmessages, etc. The images presented with the electronic message 550 canbe presented according to the user specifications. For example, if theuser specifies that a maximum of three images are to be presented withan electronic message 550, the number of images presented with theelectronic message can be limited to a maximum of three.

When selecting images to present with an electronic message 550, theimage selection application 514 can compare the meaning of one or moreword groups contained in the electronic message 550, which meat the atleast one criterion, to the context of the images contained in the imagedatabase 520. Responsive to the image selection application 514identifying at least a partial match between a meaning and context of animage, the image selection application 514 can select that image. Forexample, if the meaning of a word group is determined to be “contactsupport” and the context of an image is “support,” “contact,” “contactsupport,” or an acronym or synonym of any of these, the image selectionapplication 514 can identify that image as a candidate for selection. Inone aspect, images that are identified as candidates for selection canbe limited to those that meet user defined and/or user selectedcriteria.

If only one candidate for selection is identified, that image can beselected by the image selection application 514. In some cases, however,there may be more than one image identified as having context that is atleast a partial match to the meaning of the word group, and thus theremay be a plurality of candidates for selection. In such case, the imageselection application 514 can assign a value to each candidate imageindicating a level of relevance to the meaning. For example, if adetermined meaning of a word group is “contact support,” the imageselection application 514 can assign a first value to the word“contact,” a second value to the word “support” and a third value to theword group “contact support.” For an image in the image database inwhich the context contains the word “contact,” the image selectionapplication 514 can assign the first value to that image. For an imagein the image database in which the context contains the word “support,”the image selection application 514 can assign the second value to thatimage. For an image in the image database in which the context containsthe word group “contact support,” the image selection application 514can assign the third value to that image. The image selectionapplication 514 can rank each of the identified images based on theassigned values, and select the image having the highest value topresent with a corresponding electronic message 150.

Still, the image selection application 514 can select images from theimage database 520 in any other suitable manner and the presentarrangements are not limited in this regard. In one arrangement, aplurality of different algorithms embodied as computer program code,including the algorithm described above, can be available to the imageselection application 514 for use in selecting images. In such case, auser can be provided an option to choose which algorithm is to beexecuted to select images, and the user selected algorithm can beexecuted to select the images.

In one aspect of the present arrangements, in lieu of, or in additionto, the electronic message system 510 including the image selectionapplication 514, one or more of the client devices 530-534 can execute arespective image selection application 560, 562, 564. Accordingly, eachof the client devices 530-534 can perform the processes previouslydescribed with respect to execution of the image selection application514 by the electronic message system 510 and described with respect toFIGS. 1-4. As noted, such processes can include identifying word groupsin electronic messages 550 that satisfy at least one criterion,automatically selecting images that correspond to the meaning of theword groups, and presenting the selected images with the electronicmessages 550 to respective users. The electronic messages 550 can begenerated or received by the client devices 530-534. The client devices530-534 can be communicatively linked to the image database 520 via thenetwork(s) 540.

FIG. 6 is a flow chart illustrating an example of a method 600 ofpresenting with electronic messages images selected for respective wordgroups contained in the electronic messages. At step 605, a plurality ofword groups that satisfy at least one criterion, each word groupcomprising at least one word, can be identified in an electronicmessage. At step 610, for each word group that satisfies the at leastone criterion, at least a first image corresponding to a meaning of theword group can be selected, using a processor, from a plurality ofimages. At step 615, each image selected for each respective word groupthat satisfies the at least one criterion can be presented with theelectronic message. In one arrangement, a plurality of electronicmessages can be presented together, for example in an activity stream.In such an arrangement, steps 605-615 can be performed for each of theplurality of electronic messages.

FIG. 7 is a block diagram illustrating example architecture for aprocessing system 700. The processing system 700 can be, for example,the electronic message system 510 or a client device 530-534 of FIG. 3.

The processing system 700 can include at least one processor 705 (e.g.,a central processing unit) coupled to memory elements 710 through asystem bus 715 or other suitable circuitry. As such, the processingsystem 700 can store program code within the memory elements 710. Theprocessor 705 can execute the program code accessed from the memoryelements 710 via the system bus 715. It should be appreciated that theprocessing system 700 can be implemented in the form of any systemincluding a processor and memory that is capable of performing thefunctions and/or operations described within this specification. Forexample, the processing system 700 can be implemented as a server, aplurality of communicatively linked servers, a workstation, a desktopcomputer, a mobile computer, a tablet computer, a laptop computer, anetbook computer, a smart phone, a personal digital assistant, a set-topbox, a gaming device, a network appliance, and so on.

The memory elements 710 can include one or more physical memory devicessuch as, for example, local memory 720 and one or more bulk storagedevices 725. Local memory 720 refers to random access memory (RAM) orother non-persistent memory device(s) generally used during actualexecution of the program code. The bulk storage device(s) 725 can beimplemented as a hard disk drive (HDD), solid state drive (SSD), orother persistent data storage device. The processing system 700 also caninclude one or more cache memories (not shown) that provide temporarystorage of at least some program code in order to reduce the number oftimes program code must be retrieved from the bulk storage device 725during execution.

One or more network adapters 730 can be coupled to processing system 700to enable the processing system 700 to become coupled to other systems,computer systems, remote printers, and/or remote storage devices throughintervening private or public networks. Modems, cable modems,transceivers, and Ethernet cards are examples of different types ofnetwork adapters 730 that can be used with the processing system 700.Input/output (I/O) devices (not shown), such as a display, touchscreen,keyboard, pointing device, etc. optionally can be coupled to theprocessing system 700.

As pictured in FIG. 7, the memory elements 710 can store the componentsof the processing system, namely an operating system 735 and the imageselection application 514 (or image selection application 560-564). Theimage selection application 514 (or image selection application 560-564)can be executed by the processor 705 to implement the processesdescribed herein, including the method 600 of FIG. 6. In an arrangementin which the processing system 700 is the electronic message system 510of FIG. 3, the memory elements also can store the electronic messageapplication 512 (shown in FIG. 3). In an arrangement in which theprocessing system 700 is a client device 530-534 of FIG. 3, the memoryelements also can store one or more web browsers, electronic messagingclients and/or the like. Being implemented in the form of executableprogram code, these components of the processing system 700 can beexecuted by the processing system 700 and, as such, can be consideredpart of the processing system 700. Moreover, the operating system 735and the image selection application 514 (or image selection application560-564), as well as other components, are functional data structuresthat impart functionality when employed as part of the processing system700 of FIG. 7.

While the disclosure concludes with claims defining novel features, itis believed that the various features described herein will be betterunderstood from a consideration of the description in conjunction withthe drawings. The process(es), machine(s), manufacture(s) and anyvariations thereof described within this disclosure are provided forpurposes of illustration. Any specific structural and functional detailsdescribed are not to be interpreted as limiting, but merely as a basisfor the claims and as a representative basis for teaching one skilled inthe art to variously employ the features described in virtually anyappropriately detailed structure. Further, the terms and phrases usedwithin this disclosure are not intended to be limiting, but rather toprovide an understandable description of the features described.

For purposes of simplicity and clarity of illustration, elements shownin the figures have not necessarily been drawn to scale. For example,the dimensions of some of the elements may be exaggerated relative toother elements for clarity. Further, where considered appropriate,reference numbers are repeated among the figures to indicatecorresponding, analogous, or like features.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a,” “an,” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “includes,”“including,” “comprises,” and/or “comprising,” when used in thisdisclosure, specify the presence of stated features, integers, steps,operations, elements, and/or components, but do not preclude thepresence or addition of one or more other features, integers, steps,operations, elements, components, and/or groups thereof.

Reference throughout this disclosure to “one embodiment,” “anembodiment,” or similar language means that a particular feature,structure, or characteristic described in connection with the embodimentis included in at least one embodiment described within this disclosure.Thus, appearances of the phrases “in one embodiment,” “in anembodiment,” and similar language throughout this disclosure may, but donot necessarily, all refer to the same embodiment.

The term “plurality,” as used herein, is defined as two or more thantwo. The term “another,” as used herein, is defined as at least a secondor more. The term “coupled,” as used herein, is defined as connected,whether directly without any intervening elements or indirectly with oneor more intervening elements, unless otherwise indicated. Two elementsalso can be coupled mechanically, electrically, or communicativelylinked through a communication channel, pathway, network, or system. Theterm “and/or” as used herein refers to and encompasses any and allpossible combinations of one or more of the associated listed items. Itwill also be understood that, although the terms first, second, etc. maybe used herein to describe various elements, these elements should notbe limited by these terms, as these terms are only used to distinguishone element from another unless stated otherwise or the contextindicates otherwise.

The term “if” may be construed to mean “when” or “upon” or “in responseto determining” or “in response to detecting,” depending on the context.Similarly, the phrase “if it is determined” or “if [a stated conditionor event] is detected” may be construed to mean “upon determining” or“in response to determining” or “upon detecting [the stated condition orevent]” or “in response to detecting [the stated condition or event],”depending on the context.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the describedembodiments. The terminology used herein was chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein.

What is claimed is:
 1. A method comprising: identifying in a firstelectronic message a plurality of word groups that satisfy at least onecriterion, each word group comprising at least one word; for each wordgroup that satisfies the at least one criterion, automaticallyselecting, using a processor, from a plurality of images at least afirst image corresponding to a meaning of the word group; andpresenting, in a tree structure, the first electronic message in textformat and each image selected for each respective word group thatsatisfies the at least one criterion, the tree structure presenting eachselected image below text of the first electronic message and the treestructure indicating that each selected image corresponds to the firstelectronic message.
 2. The method of claim 1, further comprising:identifying in each of a plurality of additional electronic messages atleast one word group that satisfies the at least one criterion, thefirst electronic message and the plurality of additional electronicmessages contained in an activity stream; for each word group of theplurality of additional electronic messages that satisfies the at leastone criterion, automatically selecting from the plurality of images atleast one additional image corresponding to a meaning of the word groupof the additional electronic message; and presenting with each of theadditional electronic message the respective additional image.
 3. Themethod of claim 1, wherein presenting with the first electronic messageeach image selected for each respective word group that satisfies the atleast one criterion comprises: presenting at least one consolidatedimage comprising at least two images selected from the plurality ofimages.
 4. The method of claim 1, wherein selecting from the pluralityof images at least the first image corresponding to a meaning of theword group comprises selecting an image that satisfies at least one userdefined or user selected criterion specifying an image type.
 5. Themethod of claim 1, wherein presenting with the first electronic messageeach image selected for each respective word group that satisfies the atleast one criterion comprises presenting with the first electronicmessage a plurality of the selected images, a total number of theplurality of the selected images presented with the first electronicmessage satisfying a user specified criteria.
 6. The method of claim 1,wherein the criterion is selected from a group of criteria consisting ofa topic criteria, a tag criteria, and an alert criteria.
 7. The methodof claim 1, further comprising: receiving, via a menu, the at least onecriterion from a user, wherein the at least one criterion specifies atype of at least the first image or desired image attributes for the atleast one image.
 8. A system, comprising: a processor programmed toinitiate executable operations comprising: identifying in a firstelectronic message a plurality of word groups that satisfy at least onecriterion, each word group comprising at least one word; for each wordgroup that satisfies the at least one criterion, automatically selectingfrom a plurality of images at least a first image corresponding to ameaning of the word group; and presenting, in a tree structure, thefirst electronic message in text format and each image selected for eachrespective word group that satisfies the at least one criterion, thetree structure presenting each selected image below text of the firstelectronic message and the tree structure indicating that each selectedimage corresponds to the first electronic message.
 9. The system ofclaim 8, the executable operations further comprising: identifying ineach of a plurality of additional electronic messages at least one wordgroup that satisfies the at least one criterion, the first electronicmessage and the plurality of additional electronic messages contained inan activity stream; for each word group of the plurality of additionalelectronic messages that satisfies the at least one criterion,automatically selecting from the plurality of images at least oneadditional image corresponding to a meaning of the word group of theadditional electronic message; and presenting with each of theadditional electronic message the respective additional image.
 10. Thesystem of claim 8, wherein presenting with the first electronic messageeach image selected for each respective word group that satisfies the atleast one criterion comprises: presenting at least one consolidatedimage comprising at least two images selected from the plurality ofimages.
 11. The system of claim 8, wherein selecting from the pluralityof images at least the first image corresponding to a meaning of theword group comprises selecting an image that satisfies at least one userdefined or user selected criterion specifying an image type.
 12. Thesystem of claim 8, wherein presenting with the first electronic messageeach image selected for each respective word group that satisfies the atleast one criterion comprises presenting with the first electronicmessage a plurality of the selected images, a total number of theplurality of the selected images presented with the first electronicmessage satisfying a user specified criteria.
 13. The system of claim 8,wherein the criterion is selected from a group of criteria consisting ofa topic criteria, a tag criteria, and an alert criteria.
 14. The systemof claim 8, the executable operations further comprising: receiving, viaa menu, the at least one criterion from a user, wherein the at least onecriterion specifies a type of at least the first image or desired imageattributes for the at least one image.
 15. A computer program productcomprising a computer readable storage medium having program code storedthereon, the program code executable by a processor to perform a methodcomprising: identifying, by the processor, in a first electronic messagea plurality of word groups that satisfy at least one criterion, eachword group comprising at least one word; for each word group thatsatisfies the at least one criterion, automatically selecting, by theprocessor, from a plurality of images at least a first imagecorresponding to a meaning of the word group; and presenting, by theprocessor, in a tree structure, the first electronic message in textformat and each image selected for each respective word group thatsatisfies the at least one criterion, the tree structure presenting eachselected image below text of the first electronic message and the treestructure indicating that each selected image corresponds to the firstelectronic message.
 16. The computer program product of claim 15, themethod further comprising: identifying in each of a plurality ofadditional electronic messages at least one word group that satisfiesthe at least one criterion, the first electronic message and theplurality of additional electronic messages contained in an activitystream; for each word group of the plurality of additional electronicmessages that satisfies the at least one criterion, automaticallyselecting from the plurality of images at least one additional imagecorresponding to a meaning of the word group of the additionalelectronic message; and presenting with each of the additionalelectronic message the respective additional image.
 17. The computerprogram product of claim 15, wherein presenting with the firstelectronic message each image selected for each respective word groupthat satisfies the at least one criterion comprises: presenting at leastone consolidated image comprising at least two images selected from theplurality of images.
 18. The computer program product of claim 15,wherein selecting from the plurality of images at least the first imagecorresponding to a meaning of the word group comprises selecting animage that satisfies at least one user defined or user selectedcriterion specifying an image type.
 19. The computer program product ofclaim 15, wherein the criterion is selected from a group of criteriaconsisting of a topic criteria, a tag criteria, and an alert criteria.20. The computer program product of claim 15, the method furthercomprising: receiving, via a menu, the at least one criterion from auser, wherein the at least one criterion specifies a type of at leastthe first image or desired image attributes for the at least one image.